Representing discourse for automatic text summarization via shallow NLP techniques

In this thesis I have addressed the problem of text summarization from a linguistic perspective. After reviewing some work in the area, I have found that many satisfactory approaches to text summarization rely on general properties of language that are reflected in the surface realization of texts. The main claim of this thesis is that some general properties of the discursive organization of texts can be identified at surface level, that they provide objective evidence to support theories about the organization of texts, and that they can be of help to improve existing approaches to text summarization. In order to provide support for this claim, I have determined which shallow cues are indicative of discourse organization, and, of these, which fall under the scope of the natural language processing techniques that are currently available for Catalan and Spanish: punctuation, some syntactical structures and, most of all, discourse markers. I have developed a framework to obtain a representation of the discursive organization of texts of interand intra-sentential scope. I have described the nature of minimal discourse units (discourse segments and discourse markers) and the relations between them, relating theoretical explanations with empirical descriptions, systematizing discursive effects as signalled by shallow cues. Based on the evidence provided by shallow cues, I have induced an inventory of basic meanings to describe the discursive function of relations between units as signalled by shallow cues. The inventory is organized in two dimensions of discursive meaning: structural (continuation and elaboration) and semantic (revision, cause, equality and context). In turn, each dimension is organized in a range of markedness, with a default meaning that is assigned to unmarked cases. I have shown that the proposed representation contributes to improve the quality of automatic summaries in two different approaches: it was integrated within a lexical chain summarizer, to obtain a representation of text that combines cohesive and coherence aspects of texts, and it was also one of the components of the analysis of text in an e-mail summarizer. Finally, some experiments with human judges indicate that this representation of texts is useful to explain how some discursive features influence the perception of relevance, more concretely, to characterize those fragments of texts that judges tend to consider irrelevant. Experiments with automatic procedures for the analysis of texts correlate well with the perception of relevance observed in human judges.